SCRAPING TENNIS WEBSITES

Objectives of the project

• Gather information from the official ATP website (rankings, player statistics…)

• Build a scrapper to extract the data (Selenium)

• Create a csv file and perform some analysis with it (Shiny)

Libraries used

library(scrapex)
library(rvest)
library(httr)
library(dplyr)
library(tidyr)
library(stringr)
library(ggplot2)
library(xml2)
library(lubridate)
library(tidyverse)
library(RSelenium)

Scraping the atp rankings

• Atp_link <- “https://www.atptour.com/es/rankings/singles?rankRange=0-100&region=all&dateWeek=Semana%20Actual&SortField=Ranking&SortAscending=True”
• The website does not let you scrape it unless an user-agent is set
• 1st attempt: atp_tables[[1]] -> did not work

ATP Ranking

• 2nd attempt
• top 300 players

xml_find_all(atp_website,"//span[@class='lastName']")

#<!-- //span: Find all <span> elements -->
#<!-- [@class='lastName']: only with the class lastName -->

Scraping the aces of each player

Aces table:

Scraping the rest of the variables

• Same process as with the aces table
• After cleaning all the tables, join by ranking number
• Additional 2 variables extracted from another website

Other scraped variables

• Wins and looses
• Points won with first serve
• % of the wins of the player when he has to serve
• % of the wins of the player when he has to receive the serve
• % of break points saved

Age + country

• Age and country of the player from another website -> ATP website was not letting us do it

age_country_link <- "https://tennisabstract.com/reports/atpRankings.html" |>
  read_html()

age_country <- age_country_link |> html_table()

atp_table <- age_country[[1]]
atp_table

# Let´s clean the table:

atp_table <- atp_table[-1, ]
atp_table <- atp_table[-1, ]
atp_table <- atp_table[, 1:4]

colnames(atp_table) <- c("Rank", "Player", "Country", "Birthdate")

# Keep only the first 300 rows
atp_table <- atp_table |> slice(1:300)

# Calculate age from birthdate
atp_table <- atp_table |> 
  mutate(Birthdate = as.Date(Birthdate),
         Age = floor(interval(Birthdate, today()) / years(1)))


# Keep only Rank, Last_Name, and Age
atp_table <- atp_table |>  select(Rank, Player, Age, Country)
atp_table$Rank <- as.numeric(atp_table$Rank)

Age + country table:

Final dataset

Usage of the data

• Remove duplicate or irrelevant columns
• Convert percentages to numeric values
• Handle missing values
• Convert data into correct format

# Delete “Matches_played” and “Total_points” columns because they have 100% of NA
df <- df %>% select(-Matches_played, -Total_points)

# Merge column data into one column, taking non-NA values
df$matches_played <- coalesce(df$Matches_played.x, df$Matches_played.y, df$Matches_played.x.x, df$Matches_played.y.y)

# Delete original columns
df <- df %>% select(-Matches_played.x, -Matches_played.y, -Matches_played.x.x, -Matches_played.y.y)

# Delete unwanted columns
df <- df %>% select(-Points_won, -total_points, -points_won)

Shiny app 1

• ATP Player Comparator
• Select a professional tennis player and compare their performance metrics with custom input values
• Purpose: provide players, coaches, and analysts with a way to compare their performance to elite players, helping to identify strengths and areas for improvement

Shiny app 2

• Tennis match simulator
• Two simulation modes:
  • Monte Carlo simulation (predicts match outcomes over 1,000 iterations)
  • Set-based simulation that determines the winner set by set
• Valuable tool for fans, analysts, and players who want to assess potential matchups and performance factors